Skip to content

perf: specialize OnceMap for read-mostly workloads - #241

Closed
tisonkun wants to merge 1 commit into
apache:mainfrom
tisonkun:codex/specialize-once-map
Closed

perf: specialize OnceMap for read-mostly workloads#241
tisonkun wants to merge 1 commit into
apache:mainfrom
tisonkun:codex/specialize-once-map

Conversation

@tisonkun

Copy link
Copy Markdown
Member

Summary

  • Store Pending and Ready entries directly in one hash table, with waiter coordination allocated only while a value is being initialized.
  • Remove the OnceCell and semaphore dependency from the once-map feature without adding dependencies or unsafe code.
  • Preserve retry, cancellation, panic, and detached-generation semantics, including waiters that joined an initialization before remove or discard.

Design Notes

A ready entry now retains only its key and Arc<V>. A pending entry owns an Initialization containing the transient wait set; successful publication changes that table entry in place, while failure removes it and wakes one of the joined callers to retry.

hashbrown::HashTable remains as the raw index. Once a non-Clone key has moved into the table, the initializer must locate its exact generation using a precomputed hash and Arc identity; stable std::collections::HashMap does not expose the raw-hash lookup needed to do this without duplicating the key or scanning the map.

Removing permanent coordination has a deliberate cold-path cost: successful first initialization allocates the stored Arc<V> and transitions the table entry after the computation. This favors the intended initialize-once/read-many workload rather than cold insertion throughput. Debug reports entry and pending counts so it does not call user formatting code while holding the table lock.

Benchmark

The table reports the median of four isolated runs of the current OnceMap benchmark through the Cargo benchmark harness on the same machine.

Path main This PR Result
Successful initialization, empty map 79.05 ns 115.30 ns 45.9% deterioration
32 independent cold keys 2.778 µs 3.894 µs 40.2% deterioration
32 callers, same cold key 2.266 µs 1.844 µs 18.6% optimization
Cached compute, same key, 1 thread 11.17 ns 11.17 ns effectively unchanged
Cached compute, 64 distributed keys, 1 thread 17.52 ns 12.47 ns 28.8% optimization
Cached compute, 1024 distributed keys, 1 thread 15.81 ns 11.50 ns 27.3% optimization
get, 1024 distributed keys, 1 thread 9.22 ns 7.76 ns 15.9% optimization
remove hit, 1024 entries 64.39 ns 33.95 ns 47.3% optimization

Highly contended 32-thread results varied by path within roughly a 7.6% deterioration to an 11.4% optimization, so this change does not claim a general contention win.

Validation

  • cargo x check
  • cargo x test
  • cargo x lint
  • Focused Miri tests for colliding pending entries and removal across generations

@tisonkun tisonkun closed this Aug 30, 2026
@tisonkun
tisonkun deleted the codex/specialize-once-map branch August 30, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant